Many modern vehicles include advanced driver assistance systems (ADAS) to provide automated safety and/or assisted driving functionality. Many next generation vehicles will likely include autonomous driving (AD) systems to control and navigate the vehicles independent of human interaction. ADAS and AD systems often utilize object recognition and classification, which can include analyzing sensor data to identify detection events and known objects present in data. Machine learning techniques are commonly used for object recognition and classification. A machine learning unit can be trained to recognize a specific object or class of objects based on a set of training data representing the object or class of objects. It may, however, not be possible to learn every possible variation of a specific object that could be presented in sensor data.
There are situations that are not in the training set and are therefore unknown to the system. These are called corner cases. For example, a to-be-detected object could be viewed from a new angle, partially occluded by another object, dimly illuminated, and/or otherwise obscured. To properly detect such corner cases, a typical system may need to see a large amount of training data, which is often not computationally or otherwise practically feasible. Machine learning methods relying on large training sets may also be more complex, which can result in latency at run-time when a detection event or an object is classified.
One approach to overcome these inefficiencies is sensor fusion—the coordinated use of multiple sensors. Typical sensor fusion approaches are based on object-level fusion of sensor data, which applies object identification at an early stage and using one sensor type. Data from multiple sensors can be fused, but this is typically executed on object level data, which has already been processed to identify objects included therein. At this stage, much information has been removed. For example, a partially occluded object may not be included in the data because individual sensors may filter the object out as noise or a non-event.